Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Add integration tests for Kubernetes docker-registry deployment #328

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

eaudetcobello
Copy link
Contributor

@eaudetcobello eaudetcobello commented Feb 12, 2025

Adds a test that deploys the docker-registry charm, pulls an image to the registry, tags it with a unique name, and finally tries to pull the image from the registry.

This validates that the charm sets the custom_containerd_config from charm configuration in the right place on the filesystem.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

license-eye has checked 157 files.

Valid Invalid Ignored Fixed
81 1 75 0
Click to see the invalid file list
  • tests/integration/data/test_registries/pod.yaml
Use this command to fix any missing license headers
```bash

docker run -it --rm -v $(pwd):/github/workspace apache/skywalking-eyes header fix

</details>

@eaudetcobello eaudetcobello force-pushed the KU-2526/registry-testr branch 2 times, most recently from 32a8634 to 1edb193 Compare February 12, 2025 21:28
@addyess
Copy link
Contributor

addyess commented Feb 12, 2025

@eaudetcobello can you add test_registry as another tested module for amd64

  • look in .github/workflows/integration_test.yaml and find around line 47
          modules: '["test_k8s", "test_etcd", "test_ceph", "test_upgrade"]'

add your module (test_registry) in there.

@eaudetcobello eaudetcobello marked this pull request as ready for review February 13, 2025 16:49
@eaudetcobello eaudetcobello requested a review from a team as a code owner February 13, 2025 16:50
Copy link
Member

@mateoflorido mateoflorido left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. small nit regarding a potential TypeError in the code.

Copy link
Contributor

@bschimke95 bschimke95 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly LGTM, minor comments - great work!

# Check that the units are ready
k8s_app = kubernetes_cluster.applications["k8s"]

assert k8s_app
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is a sufficient check. See

async def test_nodes_ready(kubernetes_cluster: juju.model.Model):

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, i agree with @bschimke95. you should have the same test as in test_k8s or just drop this test all together. I don't really think we need test_ready_nodes. The model is going to deploy and be active/idle when they nodes are ready. I say just drop this test altogether.


k8s_unit = kubernetes_cluster.applications["k8s"].units[0]
try:
created = create_from_yaml(api_client, yaml_objects=test_pod_manifest)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

create_from_yaml is an async function, right?
Should this be created = await create_from_yaml(...) then?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, not async

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, create_from_yaml is ruthless.

Fail 1) it gives no type-hinting.
Fail 2) the comments on the return types are ambiguous

    Returns:
        The created kubernetes API objects.

Fail 3) What it actually returns:
a List of Lists of API objects....

Give this a shot

Suggested change
created = create_from_yaml(api_client, yaml_objects=test_pod_manifest)
created.extend(*create_from_yaml(api_client, yaml_objects=test_pod_manifest))

Copy link
Contributor

@addyess addyess left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using that kubernetes api is a little bonkers at time...here's a suggestion

# Check that the units are ready
k8s_app = kubernetes_cluster.applications["k8s"]

assert k8s_app
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, i agree with @bschimke95. you should have the same test as in test_k8s or just drop this test all together. I don't really think we need test_ready_nodes. The model is going to deploy and be active/idle when they nodes are ready. I say just drop this test altogether.


k8s_unit = kubernetes_cluster.applications["k8s"].units[0]
try:
created = create_from_yaml(api_client, yaml_objects=test_pod_manifest)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, create_from_yaml is ruthless.

Fail 1) it gives no type-hinting.
Fail 2) the comments on the return types are ambiguous

    Returns:
        The created kubernetes API objects.

Fail 3) What it actually returns:
a List of Lists of API objects....

Give this a shot

Suggested change
created = create_from_yaml(api_client, yaml_objects=test_pod_manifest)
created.extend(*create_from_yaml(api_client, yaml_objects=test_pod_manifest))

Copy link
Contributor

@bschimke95 bschimke95 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of minor things, otherwise LGTM

@@ -44,7 +43,7 @@ jobs:
builder-label: ubuntu-22.04
tester-arch: AMD64
tester-size: xlarge
modules: '["test_k8s", "test_etcd", "test_ceph", "test_upgrade", "test_external_certs"]'
modules: '["test_k8s", "test_etcd", "test_ceph", "test_upgrade", "test_external_certs", "test_registry"]'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@addyess why do we need to explicitly list each test? That feels like it just waits to be forgotten.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The operator workflows run a unique gh runner for every item in this list. We run all the tests on a unique runner for these 5 test modules b/c they deploy 5 different SKUs. If we pushed the registry tests into another module, they wouldn't have to be added to this list. you COULD push this into the test_k8s module. Or leave it as is and add this modules here.

we could detect this list i suppose. Not good for this PR though

from . import helpers

# This pytest mark configures the test environment to use the Canonical Kubernetes
# bundle with ceph, for all the test within this module.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment does not fit


# Create a pod that uses the busybox image from the custom registry
# Image: {docker_registry_ip}:5000/busybox:latest
test_pod_manifest = list(yaml.safe_load_all(_get_data_file_path("pod.yaml").open()))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's inline this path. It is only used once.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
test_pod_manifest = list(yaml.safe_load_all(_get_data_file_path("pod.yaml").open()))
test_pod_manifest = list(yaml.safe_load_all(TEST_DATA_PATH.open()))

Comment on lines +32 to +34
def _get_data_file_path(name) -> Path:
"""Retrieve the full path of the specified test data file."""
return Path(__file__).parent / "data" / "test_registries" / name
Copy link
Contributor

@addyess addyess Feb 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, this can just be a CONSTANT

Suggested change
def _get_data_file_path(name) -> Path:
"""Retrieve the full path of the specified test data file."""
return Path(__file__).parent / "data" / "test_registries" / name
TEST_DATA_PATH = Path(__file__).parent / "data/test_registries/pod.yaml"

Copy link
Contributor

@addyess addyess left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking better

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants